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United  States  General  Accounting  Office 
Washington,  D.C.  20548 


October  4,  2001 

The  Honorable  Henry  A.  Waxman 
Ranking  Minority  Member 
Committee  on  Government  Reform 
House  of  Representatives 

The  Honorable  Carolyn  B.  Maloney 
House  of  Representatives 

Nonresponse  follow-up  was  the  most  expensive  and  labor-intensive  of  all 
Census  2000  operations.  According  to  the  Bureau  of  the  Census,  it  cost 
$1.2  billion  (about  29  percent  of  the  $4.1  billion  spent  on  deceimial 
activities  in  fiscal  year  2000)  and  required  over  500,000  enumerators  to 
obtain  census  information  from  about  42  million  nonresponding 
households  in  under  10  weeks.  Because  of  this  colossal  workload,  even 
small  variations  in  productivity  can  have  significant  cost  implications.  For 
example,  if  enumerators  had  needed  as  little  as  half  a  day  more  to  complete 
their  workloads,  it  would  have  added  over  2  million  staff  hours  and  at  least 
$16  million  to  the  cost  of  the  operation,  assuming  everyone  worked  at  the 
Bureau’s  minimum  pay  rate  of  about  $8.25  per  hour.^  Not  surprisingly, 
workload  and  enumerator  productivity  have  historically  been  two  of  the 
largest  drivers  of  census  costs,  and  the  Bureau  developed  its  budget  for  the 
2000  Census  using  a  model  that  contained  key  assumptions  about  these 
two  variables. 

In  our  January  2001  response  to  your  request  for  information  on 
enumerator  productivity,  we  reported  that  productivity  data  for  the  2000 
Census  was  unavailable  because  the  Bureau  had  not  yet  assessed  its 
reliability.^  Since  then,  the  Bureau  completed  its  reliability  assessment  and 
made  certain  refinements.  As  agreed  with  your  offices,  this  report  follows 
up  on  that  earlier  request  and  presents  information  on  (1)  enumerator 
productivity  rates  by  type  of  local  census  office  and  (2)  the  Bureau’s 
methodology  for  refining  the  productivity  data. 


'Enumerator  wage  rates  ranged  from  about  $8.25  to  $18.50  depending  on  location. 

‘^Decennial  Censuses:  Historical  Data  on  Enumerator  Productivity  Are  Limited  (GAO-01- 
208R,  Jan.  5,  2001). 
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We  obtained  information  on  enumerator  productivity  rates  by  analyzing 
Bureau  productivity  data  and  the  underlying  assumptions  used  to  calculate 
them.  To  obtain  information  on  the  Bureau’s  methodology  for  refining  the 
data  and  factors  that  could  improve  the  collection  and  analysis  of 
productivity  data  in  the  future,  we  interviewed  officials  from  the  Bureau’s 
Decennial  Management  Division  and  reviewed,  but  did  not  audit,  relevant 
Bureau  documents.  On  August  20,  2001,  we  requested  comments  on  a  draft 
of  this  report  from  the  Secretary  of  Commerce.  In  his  September  18,  2001, 
written  response,  the  Secretary  of  Commerce  informed  us  that  Department 
of  Commerce  and  Bureau  of  the  Census  officials  generally  agreed  with  the 
report  and  had  no  specific  comments  on  its  content  or  recommendations. 
We  performed  our  work  from  April  through  July  2001,  in  Washington,  D.C., 
in  accordance  with  generally  accepted  government  auditing  standards. 


Results  in  Brief  Nationally,  enumerators  completed  their  nonresponse  follow-up  workload 

at  a  rate  of  1.04  housing  units  per  hour — slightly  exceeding  the  Bureau’s 
expected  rate  of  1.03  housing  units  per  hour.  Productivity  varied  for  the 
four  primary  types  of  local  census  offices,  ranging  from  0.90  housing  units 
per  hour  in  inner-city  and  urban  areas,  to  1.10  cases  per  hour  in  rural  areas. 

In  refining  the  data  thus  far,  the  Bureau  corrected  for  what  it  believes  was 
the  most  significant  discrepancy:  a  misclassification  of  certain  employees’ 
time  charges  that  overstated  the  number  of  hours  worked  by  nonresponse 
follow-up  enumerators  and  understated  enumerator  production  rates.  The 
Bureau  has  not  yet  made  any  adjustments  for  a  second  problem  identified 
in  the  data-employees  who  worked  on  both  earlier  census  operations  and 
nonresponse  follow-up  and  who  charged  their  time  to  codes  for  those 
earlier  activities  rather  than  nonresponse  follow-up.  Better  application  of 
the  Bureau’s  quality  assurance  procedures  could  have  produced  more 
reliable  data  initially  and  eliminated  the  need  to  correct  the  data  once 
nonresponse  follow-up  was  completed. 

Accurate  productivity  information  is  important  for  future  planning 
decisions.  At  the  same  time,  the  complexity  and  importance  of 
nonresponse  follow-up  requires  that  the  Bureau  place  a  premium  on 
completing  the  enterprise  according  to  its  operational  plan.  Nevertheless, 
with  proper  planning,  the  two  functions  could  be  complementary,  since  the 
Bureau’s  personnel/payroll  and  management  information  systems  already 
collect  the  raw  data  needed  to  assess  enumerator  productivity.  Therefore, 
as  the  Bureau  plans  for  2010  nonresponse  follow-up,  to  ensure  that  the 
Bureau  develops  a  baseline  of  reliable  productivity  data  for  informing 
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future  planning  decisions  consistent  with  the  operational  demands  of 
nonresponse  follow-up,  we  recommend  that  the  Secretary  of  Commerce 
direct  the  Bureau  to  (1)  determine  the  productivity  measures  needed  to 
gauge  nonresponse  follow-up  and  (2)  design  procedures,  information 
systems,  and  quality  control  mechanisms  to  reliably  capture  and  analyze 
those  measures  in  accordance  with  nonresponse  follow-up’s  operational 
requirements. 


Background 


In  our  January  2001  review,  we  described  how  the  Bureau  planned  to 
assess  and  refine  its  data  on  enumerator  productivity  because  a  significant 
number  of  individuals  on  enumerator  applicant  lists  at  some  local  census 
offices  were  hired  instead  as  crew  leader  assistants — a  different  position.^ 
In  some  instances,  the  position  change  was  not  reflected  in  the  Bureau’s 
personnel/payroll  system.  To  the  extent  that  this  occurred.  Bureau  officials 
said  that  it  would  overstate  the  number  of  hours  that  enumerators  actually 
worked  and  understate  productivity  (it  did  not,  however,  affect  actual 
payments  to  employees  because  enumerators  and  crew  leader  assistants 
were  paid  at  the  same  wage  rate).  Bureau  officials  also  had  questions 
concerning  the  extent  to  which  enumerators  who  worked  on  more  than 
one  census  operation  charged  the  codes  for  these  earlier  operations  rather 
than  the  code  for  nonresponse  follow-up.  Our  review  also  noted  that 
enumerator  productivity  rates  could  not  be  calculated  for  the  1940  through 
1990  censuses  because  needed  data  on  staffing  levels  and  hours  worked 
were  unavailable,  incomplete,  or  not  comparable.^ 

Because  productivity  information  will  be  important  for  informing  the 
Bureau’s  planning  and  budgeting  processes  for  the  next  national  head 
count  in  2010,®  we  recommended  that  the  Secretary  of  Commerce  ensure 
that  the  Bureau,  in  refining  its  productivity  figures,  identify  the  extent  and 
nature  of  any  data  anomalies,  the  impact  they  have  on  data  quality,  and  the 
extent  to  which  the  data  can  be  compared  by  type  of  local  census  office. 
To  help  ensure  the  comparability  of  data  for  the  2000  and  future  censuses, 
we  also  recommended  that  the  Bureau  fully  document  how  it  calculates 
enumerator  productivity  rates,  and  report  the  data  by  type  of  local  census 


'^GAO-01-208R. 

^GAO-01-208R. 

'^GAO-01-208R. 
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office.  In  response  to  our  recommendations,  as  part  of  its  analysis  of 
enumerator  productivity  data,  the  Bureau  included  information  on 
enumerator  production  rates  at  the  local  office  and  national  levels.  This 
information  included  an  explanation  of  the  extent  and  nature  of  certain 
anomalies  in  the  data,  the  impact  they  had  on  data  quality,  and  the 
adjustments  made  for  them.  In  addition,  the  Bureau’s  analysis  included  a 
detailed  methodology  that  documented  how  these  rates  were  adjusted  and 
calculated. 


Enumerator 
Productivity  Rates 


The  Bureau  issued  preliminary  enumerator  productivity  data  in  May  2001. 
Nationally,  enumerators  completed  1.04  housing  units  per  hour — slightly 
exceeding  the  1.03  housing  units  per  hour  that  the  Bureau  estimated  for 
budgetary  purposes.  The  Bureau  calculated  productivity  by  dividing  its 
nonresponse  follow-up  workload  (42.4  million  housing  units  nationally)  by 
enumerator  production  hours  (40.7  million  hours  nationally).  The  Bureau 
derived  this  information  from  its  personnel/payroll  and  management 
information  systems. 

Enumerator  production  hours  refer  to  the  total  time  enumerators  spent  in 
the  field  collecting  data  and  meeting  with  supervisors,  and  covers  the 
actual  duration  of  the  operation  (April  24  through  July  2,  2000).  The  hours 
exclude  time  spent  in  training  and  conducting  follow-up  activities  after  the 
Bureau  completed  the  initial  operation.  The  Bureau  also  said  it  excluded 
subsequent  fieldwork  that  it  believes  was  inadvertently  charged  to  the 
nonresponse  follow-up  code  after  its  completion.  Bureau  officials  noted 
that  the  productivity  rates  are  subject  to  minor  change  as  the  Bureau 
further  refines  its  data,  in  part  by  conducting  a  comprehensive  analysis  of 
payroll  and  personnel  data. 
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According  to  the  Bureau,  productivity  varied  by  type  of  local  office¬ 
ranging  from  0.90  to  1.21  housing  units  per  hour.  For  the  2000  Census,  the 
Bureau  had  four  primary  types  of  local  census  offices  (referred  to  as  types 
A,  B,  C,  and  D)-which  differed  by  enumeration  methods  used  and 
geographic  makeup.  TVpe  A  offices,  located  in  the  hardest-to-enumerate, 
inner-city,  and  urban  areas,  used  mailout/mailback  and  urban/update  leave 
enumeration  methodologies.  ®  TVpe  B  offices,  located  in  urban  and 
metropolitan  areas,  also  used  mailout/mailback  and  urban/update  leave 
methodologies.  TVpe  C  offices,  located  in  suburban  areas,  small  and 
medium-size  cities,  towns,  and  rural  areas,  used  update/leave, 
mailout/mailback,  and  rural  update/enumerate  methodologies.^  IVpe  D 
offices,  located  in  more  rural  areas,  used  list/enumerate,  update/leave,  and 
some  mailout/mailback  methodologies.®  TVpe  E  offices — a  fifth  office 
type — ^were  located  in  Puerto  Rico  and  used  update/leave  methodology. 

As  shown  in  table  1,  according  to  the  Bureau,  actual  enumerator 
productivity  was  20  percent  higher  than  expected  at  the  Bureau's  urban  or 
type  A  offices.  Productivity  was  about  11  percent  lower  than  expected  at 
rural  or  type  D  offices.  A  Bureau  official  told  us  that,  as  part  of  its 
evaluation  of  the  2000  Census,  the  Bureau  is  studying  reasons  for  these 
variations. 


®For  the  mailout/mailback  methodology,  the  Bureau  used  U.S.  Postal  Service  (USPS)  letter 
carriers  to  deliver  questionnaires  to  the  vast  majority  of  housing  units  that  had  city-style 
addresses  (house  number  and  street  name)  for  household  members  to  mail  back.  For  the 
urban  update/leave  methodology,  in  pre-identified  census  blocks  in  urban  areas-where  the 
USPS  might  usually  deliver  a  quantity  of  questionnaires  to  a  building  lobby-enumerators 
delivered  questionnaires  to  each  unit  and  updated  the  address  list. 

'^The  Bureau  conducted  an  update/leave  methodology  in  areas  with  primarily  non-city-style 
addresses.  For  this  methodology,  enumerators  delivered  questionnaires  to  housing  units 
and  updated  their  address  list  in  their  assignment  areas  at  the  same  time.  For  the  rural 
update/enumerate  operation,  in  pre-identified  census  blocks,  enumerators  canvassed  an 
area,  updated  the  address  list  and  associated  maps,  and  completed  census  questionnaires 
for  all  occupied  and  vacant  housing  units. 

®For  the  list/enumerate  methodology,  in  very  remote  or  very  sparsely  populated  areas, 
enumerators  visited  every  household  to  update  census  maps,  conduct  interviews,  and  list 
each  address  or  location. 
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Table  1 :  Enumerator  Productivity  During  Nonresponse  Follow-up  Varied  by  Type  of  Local  Census  Office 


Office  type 

Total 

offices 

Total  housing 
unit  workload 

Enumerator 

production 

hours^ 

Budgeted 
housing  units 
per  hour 

Actual  housing 
units  per  hour^ 

Difference 
between 
budgeted 
and  actual 
productivity 

Percentage 

difference 

National  level 

520 

42,382,492 

40,698,936 

1.03 

1.04 

0.01 

0.97% 

Type  A 

102 

6,347,900 

7,078,897 

0.75 

0.90 

0.15 

20 

Type  B 

51 

4,080,754 

4,012,296 

1.11 

1.02 

(0.09) 

(8.1) 

Type  C 

316 

28,008,736 

26,073,280 

1.11 

1.07 

(0.04) 

(3.6) 

Type  D 

42 

3,247,754 

2,960,354 

1.24 

1.10 

(0.14) 

(11.3) 

Type  E 

9 

697,348 

574,109 

N/A 

1.21 

N/A 

N/A 

N/A  =  Data  was  not  available  from  the  Bureau  at  the  time  of  our  review. 
®Preliminary  data. 

Source:  U.S.  Bureau  of  Census  data. 


Bureau  officials  cautioned  against  comparing  the  productivity  of  individual 
offices  because  of  location-specific  circumstances.  The  officials  said  that 
the  data  are  more  reliable  when  aggregated  by  local  census  office  type  and 
at  the  national  level. 

For  the  1990  Census,  the  Bureau  reported  that  enumerators  completed  1.56 
housing  units  per  hour.  However,  Bureau  officials  said  the  1990  figure 
should  not  be  directly  compared  to  the  2000  rate  because  the  Bureau  does 
not  have  documentation  on  how  the  1990  figure  was  calculated,  and  thus 
does  not  know  if  a  direct  comparison  would  be  valid. 


How  the  Bureau 
Refined  Its 
Productivity  Data 


According  to  Bureau  officials,  the  Bureau  made  a  “coarse”  adjustment  for 
what  it  considered  to  be  the  most  significant  anomaly-the  misclassification 
of  crew  leader  assistant  hours.  As  noted  earlier,  some  local  census  offices 
hired  a  substantial  number  of  crew  leader  assistants  from  lists  of  applicants 
for  enumerator  positions.  In  some  cases,  the  position  change  was  not 
entered  into  the  Bureau’s  personneFpayroll  system.  To  the  extent  this 
happened.  Bureau  officials  said  that  it  would  overstate  the  number  of  hours 
that  enumerators  actually  worked  and  thus  understate  enumerator 
productivity. 

To  adjust  for  the  misclassified  crew  leader  assistant  hours,  the  Bureau 
analyzed  productivity  rates  to  distinguish  between  those  hours  that 
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belonged  to  crew  leader  assistants  and  those  hours  that  belonged  to 
enumerators.  Based  on  its  analysis  of  daily  productivity  rates  at  six  local 
census  offices  (three  that  had  reliable  crew  leader  assistant  data  and  three 
with  no  crew  leader  assistant  data),  employees  with  a  production  rate 
greater  or  equal  to  0.2  cases  per  hour  were  considered  to  be  enumerators, 
and  their  noncase  hours  (production  in  which  no  households  were 
completed)  were  included  in  the  Bureau’s  adjusted  production  hour 
variable.  Employees  with  a  production  rate  less  than  0.2  cases  per  hour 
were  considered  to  be  crew  leader  assistants,  and  their  noncase  hours 
were  excluded  from  the  adjusted  production  hour  variable. 

Compared  to  the  original,  unadjusted  data,  using  the  refined  figures, 
enumerators  completed  nonresponse  follow-up  in  40.7  million  production 
hours  versus  44.3  million  production  hours,  a  difference  of  3.6  million 
production  hours  or  8.1  percent.  This  revised  estimate  of  production  hours 
could  be  important  for  future  budgeting  and  planning  purposes.  For 
example,  using  the  lowest  enumerator  wage  rate  of  about  $8.25  per  hour, 
the  3.6  million  production  hour  difference  would  change  the  Bureau’s  cost 
estimates  by  about  $29.7  million. 

The  Bureau  has  not  yet  made  any  corrections  for  a  second  problem 
identified  in  its  productivity  data:  employees  who  worked  on  earlier  census 
operations  who  continued  to  charge  their  time  to  those  codes  rather  than 
nonresponse  follow-up.  According  to  Bureau  officials,  the  extent  and 
impact  of  this  problem  is  unknown. 

With  both  problems,  better  application  of  quality  assurance  procedures  at 
the  time  the  productivity  data  were  collected  could  have  produced  more 
reliable  data  initially,  and  eliminated  the  need  to  correct  the  data  later  in 
the  census  cycle.  Indeed,  enumerators  charged  codes  other  than 
nonresponse  follow-up  despite  the  fact  that  supervisors  were  to  review 
payroll  forms  to  ensure  that  enumerators  entered  the  correct  task  codes. 


Conclusions  Accurate  productivity  information  is  important  for  gauging  the 

performance  of  nonresponse  follow-up,  validating  planning  assumptions, 
preparing  and  justifying  budgets,  and  devising  more  cost-effective  census¬ 
taking  techniques  for  the  future.  However,  in  past  censuses,  the  Bureau 
has  encountered  problems  obtaining  complete  and  comparable 
productivity  data,  and  the  Bureau  appears  to  have  repeated  this  pattern  in 
2000.  As  a  result,  the  Bureau  needed  to  spend  additional  resources  to 
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refine  the  information,  and  even  then,  the  adjustment  was  coarse  and 
addressed  just  one  of  the  two  known  problems. 

At  the  same  time,  given  the  size  and  decentralization  of  nonresponse 
follow-up,  a  certain  amount  of  procedural  error  and  imreliable  data  is 
unavoidable.  Moreover,  because  of  the  complexity  and  importance  of 
nonresponse  follow-up,  it  is  important  for  the  Bureau  to  emphasize 
completing  the  endeavor  according  to  its  operational  plan. 

As  the  Bureau  develops  nonresponse  follow-up  plans  for  the  2010  Census,  a 
key  challenge  will  be  developing  systems  and  procedures  for  collecting 
reliable  productivity  data  needed  for  evaluation  and  planning,  without 
detracting  from  higher-priority  operational  demands.  However,  these  goals 
are  not  necessarily  incompatible.  In  fact,  as  the  Bureau’s  persoimel/payroll 
and  management  information  systems  already  collect  the  raw  data  needed 
to  assess  enumerator  productivity,  with  proper  planning,  the  two  functions 
can  be  complementary.  Consequently,  it  will  be  important  for  the  Bureau 
to  (1)  determine  the  productivity  information  it  needs  to  evaluate 
nonresponse  follow-up  during  the  2010  decennial  and  plan  for  future 
censuses  and  (2)  integrate  those  evaluative  requirements  into  its 
operational  plans  and  management  information  systems. 


Recommendations 
Executive  Action 


for 


To  ensure  that  the  Bureau  develops  a  baseline  of  reliable  productivity  data 
for  evaluating  nonresponse  follow-up  and  informing  future  planning 
decisions,  we  recommend  that  the  Secretary  of  Commerce  direct  the 
Bureau  to,  as  part  of  its  planning  effort  for  the  2010  Census,  (1)  determine 
the  productivity  measures  needed  to  assess  nonresponse  follow-up  and 
(2)  design  procedures,  information  systems,  and  quality  control 
mechanisms  to  capture  and  analyze  those  measures  consistent  with 
nonresponse  follow-up’s  operational  requirements.  Possible  measures 
include  average  number  of  hours  worked  by  enumerators,  cases  completed 
per  production  hour,  and  number  of  cases  completed  per  enumerator.  The 
data  should,  at  a  minimum,  support  the  analysis  of  variation  in  these 
measures  by  census  region  and  type  of  local  census  office  (type  A,  B,  C,  D, 
or  E). 


We  are  sending  copies  of  this  report  to  the  Chairman  of  the  House 
Committee  on  Government  Reform;  Chairman  and  Ranking  Minority 
Member  of  the  Subcommittee  on  the  Census,  House  Committee  on 
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Government  Reform;  Secretary  of  Commerce;  and  Acting  Director  of  the 
Bureau  of  the  Census.  Copies  will  be  made  available  to  others  upon 
request.  Robert  Goldenkoff  and  Victoria  E.  Miller  made  major 
contributions  to  this  letter.  If  you  have  any  questions  concerning  this  letter, 
please  contact  me  at  (202)  512-6806. 


J.  Christopher  Mihm 
Director 
Strategic  Issues 


(450066) 
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GAO’S  Mission 

The  General  Accounting  Office,  the  investigative  arm  of  Congress,  exists  to 
support  Congress  in  meeting  its  constitutional  responsibilities  and  to  help 
improve  the  performance  and  accoimtability  of  the  federal  government  for  the 
American  people.  GAO  examines  the  use  of  public  funds;  evaluates  federal 
programs  and  policies;  and  provides  analyses,  recommendations,  and  other 
assistance  to  help  Congress  make  informed  oversight,  policy,  and  funding 
decisions.  GAO’s  commitment  to  good  government  is  reflected  in  its  core  values 
of  accountability,  integrity,  and  reliability. 

Obtaining  Copies  of 
GAO  Reports  and 
Testimony 

The  fastest  and  easiest  way  to  obtain  copies  of  GAO  documents  is  through  the 
Internet.  GAO’s  Web  site  (www.gao.gov)  contains  abstracts  and  full-text  files  of 
current  reports  and  testimony  and  an  expanding  archive  of  older  products.  The 
Web  site  features  a  search  engine  to  help  you  locate  documents  using  key  words 
and  phrases.  You  can  print  these  documents  in  their  entirety,  including  charts  and 
other  graphics. 

Each  day,  GAO  issues  a  list  of  newly  released  reports,  testimony,  and 
correspondence.  GAO  posts  this  list,  known  as  “Today’s  Reports,”  on  its  Web  site 
daily.  The  list  contains  links  to  the  full-text  document  files.  To  have  GAO  E-mail 
this  list  to  you  every  afternoon,  go  to  our  home  page  and  complete  the  easy-to-use 
electronic  order  form  found  under  “To  Order  GAO  Products.” 

Order  by  Mail  or  Phone 

The  first  copy  of  each  printed  report  is  free.  Additional  copies  are  $2  each.  A 
check  or  money  order  should  be  made  out  to  the  Superintendent  of  Documents. 
GAO  also  accepts  VISA  and  Mastercard.  Orders  for  100  or  more  copies  mailed  to  a 
single  address  are  discounted  25  percent.  Orders  should  be  sent  to: 

U.S.  General  Accounting  Office 

P.O.  Box  37050 

Washington,  D.C.  20013 

To  order  by  Phone:  Voice:  (202)  512-6000 

TDD:  (301)  413-0006 

Fax:  (202)  258-4066 

Visit  GAO’s  Document 
Distribution  Center 

GAO  Building 

Room  1100,  700  4th  Street,  NW  (comer  of  4th  and  G  Streets,  NW) 

Washington,  D.C.  20013 

To  Report  Fraud, 
Waste,  and  Abuse  in 
Federal  Programs 

Contact: 

Web  site:  www.gao.gov/fraudnet/fraudnet.htm. 

E-mail:  fraudnet@gao.gov,  or 

1-800-424-5454  (automated  answering  system). 

United  States 
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Washington,  D.C.  20548-0001 
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